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ABSTRACT 


Let  (B^X^) , . . . ,  (®n»xn)  be  ft  simple  samples  drawn 

from  the  population  (6,X)  which  is  a  {1,2,..., 3}  x  R^- valued 

random  vector.  Suppose  that  X  is  known.  Let  X^  be  the 

nearest  one  among  X^,...,Xn,  from  X,  in  the  sense  of 

Euclidean  norm  or  £  -norm,  and  let  0*  be  the  0-value  paired 

n 

with  X^.  The  posterior  error  probability  is  defined  by 
Ln  *  P(6^9  |  (6^,^), . . . ,  (0n,xn) )  .  It  is  well  known  that 
ELn  *  P(0^0)  has  always  a  limit  R.  In  this  paper  it  is 
shown  that  for  any  e  >  0,  there  exist  constants  c  <  »  and 

—fen 

b  >  0  such  that  P(|Ln-R|se)  £  ce  ,  under  the  only  assumption 
that  the  marginal  distribution  of  X  is  nonatomic. 


-.cccsf;!',-;  "or 


r ,  hir's.  s\*  I  /  5 r) 


f-j  .  *■ 

'  -  *  r.f. 


1.  INTRODUCTION 


•  The  d-  poit/ir" 


it  (4*,X)  be  a  j(l,2, .  -\»s)  x  ^-valued  random  vector 


and ^)), . . .  be  iicL  samples  dravm  ^from  (f  ,X)  . 

The  so-called  discrimination  problem  is  to  find  a  function 


of  X,  usually  depending  upon  <e. 


/  •  •  •  /  fs, 


ji,  which  is 


used  to  predict  the  value  of  8.  One  of  the  most  frequently 
used  approaches  is  the  nearest  neighbour  jtNtf^discrimination 


rule,  which  is  defined  as  follows:  Let 


be  a  norm  in 


R  .  Usually  we  take  this  norm  being  the  Euclidean  one  or 
iaa-nbrm.  When  X  *  x  is  given,  we  rearrange  X1,...,Xn 
according  to  the  increasing  order  of  the  distances 
| |xi  -  x  |  |  ,  namely, 

I lx„  -  x| I  s  I |X„  -  xl  I  £  ...  s  I (X„  -  xl I , 


ties  are  broken  by  comparing  indices,  for  instance,  if 
|  |xi  -  x|  |  *  |  |Xj  -  x|  1  and  i  <  j,  then  X.^  is  rearranged 
before  X  ^ .  In  view  of  the  rearrangement  of  X’s,  9i'***'9n 
are  rearranged  as  0R  , . ,.,0R  .  Let  k  s:  n  be  a  positive 

(V)  i  ** 

integer.  Then  9'  ,  called  k-NN  discrimination,  is  defined 

n 

to  be  the  value  which  has  the  biggest  frequency  among 

9_  ,...,9  .  If  such  value  is  not  unique,  then  we  randomly 

K1  *k 

take  one  such  value  with  equal  probability. 

The  probability  of  misdiscrimination  and  the  posterior 


one  are  defined  to  be 


(1.2) 


3. 

R^k)  »  P(0^kV9)  • 

and 

Ln°  “  p(0nk)^0l  (0x/Xi)  (0n,Xn))  , 

which  are  simply  called  error  probability  or  posterior 
error  probability,  respectively.  For  k  *  1,  X«  / 

I* 

R^  and  Iif^  are  simply  denoted  by  X',  0',  R_  and  L 
n  n  n  n  n  n 

respectively.  For  reviews  of  the  literature  on  this  topic, 
the  reader  is  referred  to  Cover  and  Hart  (1967),  Wagner  (1971) 
Frits  (1975),  Cover  (1968),  Devroye  (1981),  Bai  (1984). 

It  was  shown  that  whatever  the  distribution  of  (0,X) 

V 

is,  there  is  always  a  constant  R  ,  depending  upon  k,  such 

(k)  k  k 

that  lim  R '  *  R  ,  and  the  constant  R  satisfies 

n  n 

R*  <»  Rk  s  R  £  R1  s  R»  (2  -gS^R*)  •  where  R*  is  the  error 
probability  in  Bayesian  discrimination.  Let 

Pi(x)  *  P(9»i|X»x),  i  *  1,2,. ..,s.  (1.3) 

Then  it  can  be  shown  that 
S  2 

R  *  1  -  I  EP.  (X) .  (1.4 

i=l  1 

In  Chen  and  Kong  (1983) ,  it  was  proved  that  when 
S  *  2/Ln  ^  C  for  some  constant  C  if  and  only  if 

(P(9»l  ,X-x)  -  P (9*2 ,X*2) ) 2  P(0-l,X*x)  P (9*2 ,X=x)  *  0,  ( 


and  C  *  R.  This  implies  that  if  the  marginal  distribution 


F  of  X  has  no  atoms,  then  LR  *  R.  But  according  to 

Hewitt-Savage  zero-one  law  we  know  that  P(L  -*R)  «  0  or  1, 

n 

if  F  has  no  atoms.  The  problem  is  under  what  conditions 
P(Ln**-R)  *  1  is  true.  To  make  it  clear,  let  us  introduce 
the  following  notations. 

Suppose  X^,...,Xn  are  given.  Split  R^  into  n  sets 

Vnl,...,Vnn/  such  that  x  e  if  and  only  if  X^  is  the 

nearest  neighbor  of  x.  Note  that  V^,' j  -  l,2,...,n,  are 

random  sets.  By  the  definition  of  L  ,  it  is  not  difficult 

n 

to  compute  out  that 


n  s  r 

-  Z  Z  1(0. -i)  P.  (x)F(dX). 

j-1  i-1  3  JV  ,  1 


(1.6) 


It  is  easy  to  see  that  when  X^,...,Xn  are  given,  the  terms 
under  the  first  summation  in  (1.6)  is  a  sum  of  conditionally 
independent  and  bounded  random  variables,  and  the  coefficients 
have  expectations  not  exceeding  Thus  there  is  a  strong 
reason  to  believe  that  there  should  be  an  exponential  bound 
for  P(|L  -R|ze).  But  up  to  now,  as  the  author  knows,  the 
best  result  is  due  to  Fritz  (1975).  It  was  shown  that 


P  ( 1 1*  -R  as)  *  ce 
1  n 


-b/n 


under  the  assumptions  that  F  has  no  atoms  and  that  P^ (x) , 
i  *  l,2,...,s,  are  a.e.x.F.  continuous.  There  are  some 
extra  examples  to  support  the  conjecture  that  /n  in  (1.7) 


(1.7) 


could  be  improved  as  to  n.  Also,  there  is  much  evidence 

to  suggest  that  the  assumption  of  continuity  of  P^(x)'s 

could  be  abandoned  (see  Z.  D.  Bai,  X.  R.  Chen  and  6.  J. 

Chen  (1984)).  The  main  purpose  of  this  paper  is  to  solve 

these  two  problems.  In  §2,  we  shall  show  some  lemmas  and 

the  proof  of  main  result  will  be  given  in  13.  In  a 

further  paper,  we  will  give  the  necessary  and  sufficient 

conditions  for  L  -*■  R  a.s. 

n 

2.  SOME  LEMMAS 

In  the  sequel,  we  shall  need  the  following  lemmas: 

Some  of  them  are  known,  we  will  quote  them  below  without 
proof.  We  shall  only  give  the  proof  for  new  ones. 

Lemma  1.  Let  (  be  a  random  variable  with  distribution 
F (x)  (for  convenience  we  assume  that  F(x)  is  left  continuous) 
and  let  n  be  a  random  variable  uniformly  distributed  over 
the  interval  (0,1)  and  being  independent  of  £.  Then 

Z  L  F  (£)  +  [F (£+0)  -  F ( £) 3n 

is  uniformly  distributed  over  (0,1). 

The  proof  is  not  very  difficult  and  is  omitted  here. 
Lemma  2.  Let  F  be  a  nonatomic  probability  measure 
defined  on  R^  and  A  be  a  measure able  set  in  R^.  Then 
for  any  C  «  [0,F(A)3,  there  is  a  measurable  set  B  c  a  such 
that  F (B)  *  C. 


6. 


The  proof  of  this  lemma  can  be  found  in  Fritz  (1975) 
or  Loeve  (1977)  p.101. 

For  the  need  in  the  sequel,,  we  introduce  the  concept 
of  w-cone.  Let  0  be  a  point  in  R  and  A  be  a  measurable 
set  in  R^.  Then  A  is  called  an  w-cone  with  vertex  0  if 
for  any  x,  y  e  A,  | | x-y | |  <  max ( j 1 0-x | | ,  | 1 0-y | | ) . 

Lemma  3.  For  each  d,  there  is  a  positive  integer 
m  »  m(d) ,  depending  only  upon  d,  such  that  R^  can  be  split 
into  m  disjoint  w-cones  with  a  common  preassigned  vertex. 

The  proof  refers  to  Fritz  (1975) . 

Let  F  be  a  nonatomic  probability  measure  on  R^  and 

A  <=  be  a  measurable  subset  of  positive  F-measure.  Then, 

by  lemma  3,  we  can  split  Rd  into  m  w-cones  with  a  common 

preassigned  vertex.  Write  the  intersections  of  A  and  each 

m 

w-cone  as  K.  ,...,K  ,  with  U  K.  *  A.  If  F (K.  )  *  0, 
x  m  ^  t  t 

t  »  1,2, ...  ,1,  and  F(Kfc)  >  0,  t  ■  l  +  l,...,m,  by  lemma  2/ 
we  can  split  into  i  +  1  subsets  with  equal  F-measure. 

Drop  the  original  K^,...,Ki+2  and  write  the  new  l  +  1  subsets 
as  K^, . . .  Note  that  the  new  are  also  w-cones 

with  the  same  vertex.  Hence  we  obtain  the  following  lemma. 

Lemma  4.  Let  F  be  nonatomic,  A  be  of  positive  F-measure 
and  0  be  a  point  in  Rd.  Then  there  are  m  disjoint  w-cones 
K i , . . . , K  ,  having  the  same  vertex  0  and  satisfying 


U  K.  C  A 
t-1  C 

m 

E  F(K.  )  -  F  (A) 
t-1  * 

F(Kfc)  >  0,  t  -  1,2,... ,m. 

In  the  last  section,  we  defined  the  partition  Vn^  of 
Rd,  for  ramdom  points  X^,...,Xn.  We  point  out  that  for 
any  given  points  X^,...,Xn#  even  they  are  not  random,  we  can 
similarly  define  vn j »  j  *  1*2,..., n.  Also,  if  1  s  j  s  k  s  n, 
we  always  have  Vnj  =  Vkj- 
Lemma  5 1  Let  X^ , . . . , 
nonatomic  probability  measure  defined  on  Rd.  Suppose  that 
F(Vkj)  -  >  0,  j  *  1,2,. ..,k  and  that  X^+1,...,Xn  are  iid. 

ramdom  points  with  distribution  F.  Write  Unj  -  F (VRj ) .  Then 
there  are  mk  random  vauriatoles  j  -  l,2,...,k,4  -  l,2,...,m, 

satisfying  the  following  conditions, 

1)  P(UnjSAf1CjZ)  *  1'  j  “  lr2'***'k*  (2,1) 

2)  0  s  ^  q j ^ *  j  *  1,2, ... ,k,  £>  —  1, 2, . . . ,m  (2.2) 

3)  P (5 • «  ^w . >  f  ]  —  1,2, ... ,k,  i  —  1,2,... ,m) 

3  *  j  *■ 

k  ra  n-k 

*  (1-  E  E  v.0)  ,  for  0  s  w.9  s  q..,  (2.3) 

j-1  4-1  ^ 

where  q.9,  j  *  l,2,...,k,  l  -  l,2,...,m,  are  positive  constants 
3  * 

independent  of  X^+1, . . . ,XR  (may  depend  upon  X^, . . . ,X^  and  F) , 

m  k  m  k 

and  satisfy  E  q. .  »  q.(of  course,  E  S  q.  »  E  q .  -  1)  . 


X^  be  k  points  m  R  and  F  be  a 


Proof.  According  to  Lemma  4,  split  V^j  into  m  w-cones 

of  positive  F  measure,  denoted  by  Kj^,  j  =  l,2,...,k, 

1  —  1,2,...  ,m.  Write  q . .  «§  F  (K . « )  ^  0,  d  *  l,2,...,k, 

3*  3*  m 

2.  =  l,2,...,m.  It  is  evident  that  I  q • ,  =  q . ,  j  *  1,2,... 

2=1  3  *  3 

Define 


Djo  =  I lxj"xl I '  3  *  1/2, ...,k, 

D =  || X j“X^ | | ,  j  =  1,2, ...,k,  t=  k  +  1, • • • ,n, 

I.„  =  {t:  k+1  s  t  £  n,  X.  «  K. 

D  *  t  3  x, 

n.  =  *{I.0>  =  the  number  of  elements  of  I . . , 

3  x,  3  x,  j  x* 

min  D. .  if 

H:  “  teV 

«  otherwise. 

By  the  definition  of  w-cone,  X  €  V  .  n  K . „  and  X  e  K . . 

nj  3  *  u  3  X/ 

imply  that  D.  <  D..  hence  D.  <  H.0.  Therefore 
DO  jt  jo  j  2 


PtX‘Vnjl*k+l . V 


^  P(XeVn:"KjlllXk+l . V 


<5jlP(X‘V„jlX£Kji'Xk+l . V 


*  ^  qjJp<Djo<HjllX£Kjt'Xk+l . V 


I  q . .  min  F . „ (D . ^ ) , 


where  Fj£(u)  »  P (Dj0<ulXfiKjjj>)  ^  the  minimum  in  (2.4)  is 
one  if  I .  „  «  <p . 

j  *• 

Construct  n-k  iid.  random  variables  •  •  •  »nn,  which 

are  uniformly  distributed  over  the  interval  (0*1)  and  are 
independent  of  . . .  ,Xn*  Define  for  each  t  *  k  +  l,...,n, 

Gt  -  VV  +  CFj*lD5t+0)-Fji(V]V  i£  <2-5> 

Note  that  when  I.fl,  j  »  l,2,...,k,  £  *  l,2,...,m,  are  given 

j  * 

and  X.€K. „ ,  D..  has  the  same  conditional  distribution  as  D. 
t  ;j£  jt  30 

when  XcKj z  given.  Thus,  by  lemma  1,  Gt  is  uniformly  distributed 
over  (0,1).  Also,  when  I^£,  j  *  l,2,...,k,  l  »  l,2,...,m,  are 
given,  Gfc  depends  only  upon  Xfc  and  nfc,  hence  Gfc,  t  =  k+l,...,n, 
are  conditionally  independent.  Define 


J4“  Gt 


if 


(2.6) 


otherwise. 


Evidently,  (2.2)  follows  from  (2.6).  Also,  (2.1)  follows 
from  (2.4)  -  (2.6).  Finally,  we  have  for  0  s  w^  s 
j  *  1,2,. ..,k,  ^  *  1,2,... ,m. 


P(?j£2wj£'  j".1'2*--*  A-l,2,...,m) 


r  (n-k) ! 

L  k  m 

n  n  n.£: 

lj*l  4-1  J 


km  n^ 

11  11  ' 

j-1  4-1 


P(GJ_2-it,t€l.  .,  j-l,2,...,k,  4-1,2, . . .  ,m|  (I .  .  }) 


where  the  summation  unspecified  in  (2.7)  runs  over  all 

k  m 

possibilities  that  Z  Z  n .  *  n  -  k,  n . „  *  0 f  integers. 

j-1  l* 1  3* 

The  proof  of  lemma  5  is  completed. 

Lemma  6.  Let  $  be  an  a.e.  positive  function  defined  on 
the  line  and 

fql  fqk 

H  . . .  #g^)  *■  m  .  •  r  $ (w^+. . .+wk) dw^. . .dw^. 

Jo  o 

Then  we  have 

s ® ^ i * » • qv )  *  H (q, . . . ,q) , 

q1+...+qk  -  kq  (2 

qx  ^  0f... ,  qk  ^  0 

here  q  >  0  be  a  given  number. 

Proof.  Note  that  D  »  {  (q^  . . .  ,qfc)  ,  q^. .  .+q  =  kq, 

a  0,...,qk  £  0}  is  a  bounded  and  closed  set  and  that 
H(q^#...,qk)  is  continuous  on  D.  Hence  there  exists  a  point 
q°, . . . ,q°) £ 0  such  that 

sup  H  ( q .  ,.  .  .  , q.  )  *  H(q.,..., q.  )  . 


To  prove  (2.8),  we  only  need  to  prove 


q°  *  qj  *  . ..  *  q£.  By  symmetry  in  q°,...,q£,  we  only 


need  to  prove  that  q°  -  q^>  Let 


<Mx) 


r  -  r 

*  n  *  t 


<p  (x+w3+. .  ,+w^)  dw3* .  .dw^. 


Then 


qo  qo 

Q(q°,q°)  £  H(q°,q°,...,q£)  -  f  1f  2  <Mx+y)dxdy. 


o  Jo 


o  o 

rqi+q2 


<|>  ( u)  min  ( u ,  qj ,  q  2 ,  q  }  du 


o  o 

^  1  2  ip  (u)min(u,q°+q2"U)du 


o  o  o  o 
ql+q2  ql+q2 

—5 - 


_ _  „o .  o  -  o  o 

1 —  ql+q2  ql+q2 

ip(x+y)dxdy  *  Q(-=^ — •  — j-*) 


o  o 
ql+q2 


(2 


The  quality  of  (2.9)  holds  if  and  only  if  ~  — 2 — 

because  <Hu)  is  positive.  The  proof  of  lemma  6  is  completed 


Lemma  7  (Bennett,  1962,  see  Hoeffding  (1963)).  Let 

2  _„„2 


Ui . Un  be  independent  and  let  EU^  »  0,  a2  »  EUi, 


a2  *  —  S  a2.  Suppose  that  |u. |  s  b,  i  «  1,2 
n  i-1  1  1 

for  any  e  >  0, 

n 


f  •  •  •  9 


pUr  2  0.  |ae)  s  2  exp(-ne2/2 (o2+bc) }. 


n.  Then 


12. 


Lemma  8  (Hoeffding,  1963).  Let  £  ^  B(n,p),  the  binomial 
distribution  with  parameter  n  and  p.  Then  for  any  e  >  0, 

P( |~5-p|2s)s2exp{-ne2/(2p+£) >• 

3.  THE  MAIN  RESULT 

Theorem  3.1.  If  F,  the  marginal  distribution  of  X, 
has  no  atoms ,  then  for  any  e  >  0 ,  there  exist  constants  c 
and  b,  depending  upon  e  and  the  distribution  of  (9,X),  such 
that 


P  ( I  Ln~R  I  Jte )  s  ce 


-bn 


(3.1) 


Proof.  Define 


Tn  *  p<eA*0lxi'--wXn)  -  E(Ln|Xr...rXn) 


Using  the  notations  defined  in  §1  and  12,  we  have 

n  s 

and 


ns  f 

L _  »  1  -  Z  Z  I(9.»i)  P. (x) F 

3-1  i-l  j  Jvnj 


(dx) 


ns  r 

-  1  -  Z  Z  P.  (X.)  P.  (x) F 

j-i  i-i  1  ^  ivnj 


(dx) 


Hence 


P<IVTnl*2se,) 


(3.2) 


(3.3) 


(3.4) 


s  n  t 

Z  P  ( |  Z  (I(9.-i)-P.(X.))  P.  (x)F(dx)  |a2e')  (3.5) 

i-l  J-i  3  13  Jvnj  1 


Define 

-  f  P,  (x)F(dx)  s  U  .  -  f  F(dx).  (3.6 

n3  Jv  i  n  3  Jv 

nj  nj 

*n  “  ( 3* j^n,OnjSA/n},  »  (1,2, . . . ,n}\*n  (3.7 

and 

k  -  [6nl  (3.8 


where  A  and  <5  are  positive  constants  to  be  specified  later, 
and  Cx]  denotes  the  largest  integer  less  than  or  equal  to 
x.  By  lemma  5  and  lemma  6,  we  have 


k  m 


P(  Z  U  .  £e '  |  X. , . . .  ,X.  )  s  P(  Z  Z  Z..2e' 
j»l  1  K  j«l  £»1 


in  k  mxj.  J0  jo  d  j,i  M  j«l  i»i 


1  1 
f®^ 


.  *  ,  rnuc  rmx  k  m  n-k-mk  k  m 

/n-vlmki  »  •••  In(1"  2  E  WiA}  11  n  dWi£ 

(n  k  mk) .  JQ  JQ  D  ^  4ib1  3*  j»i  ^  3* 


mk 


i  (n-k-mk) !  <5E>  exp(-e- (n-k-mk) } , 


(3.9 


k  m 

where  D  =  {w. . , j*l, 2 , . . . ,k, £«1, 2, . . . ,m,  Z  Z  w.-ie'}  and 


IQ  is  the  indicator  of  the  set  D. 


j-1  £-1 


j* 


Let 


max 


E  »  {  _  _  -  . 

n  lsj,<...<j.  sn  i-1  n3 


£  U_^  ae* } 

i 


(3.1 


"  v  tb.v  i.  n.  '.i\^  ,vt. <*.--> s^rwr’. j* '«rz'w^r^s: ^Ti  •-  .  "  k.  '  ■  J'*  I »  V'  *  " '*'■.■  I  ■  1^ 


14. 


We  have  from  (3.9) 


P(En)  £  Z  P(  Z  U  .  ae') 

lsji<. . .<jksn  i-1  n3i 


<*,P<j*1'V£')  ‘  <k)EP(^1°ni2e'l3!l . V 


mk 


s  (k>  Tn&&i' !  exp{-e'  (n-k-mk)  >. 


(3.11) 


Applying  Stirling's  formula  and  recalling  k  =  [5n],  we  have 


P(En)  5  k*  TmKT * . (n-mk-k) 1  exp{-e' (1-6 (m+1) )n} 

£  [6“<S(m6)“m6(l-*(m+l)5)“(1"(m+1)<S)  D^xpt-e*  (l-(m+l)6)n} 


£  exp(-bn). 


(3.12) 


j  e'(l-(m+l)6)  and  6  e  ( 0  r  min  (e ' ,  j^-) )  f  being  such 


where  b 
that 


logC5"<S(m6)“m<S(l-(m+l)5)"(1"(m+1),5)  3  <  |  e  ’  (1-  (m+1)  6)  .  (3.13) 

Here  the  reason  that  such  5  can  be  chosen  is  based  upon  the 

fact  that  when  5  4-0,  the  left  hand  side  of  the  above  inequality 

tends  to  zero  and  the  right  hand  side  to  y  s'  >  0. 

i  c 

If  we  choose  A  >  t,  then  #{$  }  £  k,  hence  we  have 

0  n 
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P(|  £  (I(ej-i)-Pi(Xj)) J  Pi(x)F(dx) |*e') 


nj 


*  P(  £  U  ,*e') 


j£* 


nj' 


<,  P(En>  s  exp{-bn}, 


Hence 


n  f 

P  ( I  £  (iOj-D-P^Xj))  j  P±  (x)F(dx)  |  &2e ' ) 
j*l  . 


nj 


s  e“bn  +  P < |  £  (1(0, -i)  -  P. (X.))Q  , |ae') 

j£«n  j  1  3  nj 


e~bn  +  EP(|  £  (I(e.«i)  -  Pi(X,))Q  . |ae' |XL,... rXn 

j  «•  3  ^  ^ 

n 


Note  that  when  X^,...,Xn  are  given,  1(0^**'').#  j  *  l,2,...,n 
are  conditionally  independent  and  P^(Xj),  Qnj  and  depend 
only  upon  X^,...,XR.  Hence  applying  lemma  7,  we  have 


P(LZ*  (I(9j“i)  "  Pi(Xj)  )Qnj  l*e'  lxr”'*Xn) 


1«*. 


pU^j.*  U(9i’i)  '  . V 


S  2  exp{-n,(|V/2(i-  £  Q* ,  +  ~  max  Q  . )  } 

n*  n*  nj  n*  nj 


j**. 


s  2  exp{-e'/2 (1+e ' )  max  Qn ^ } 


j 


n 


s  2  exp{-(eV2(l+e’)A)n}£2e"bn, 


where  n*  =  #{$  },  and  b  is  a  positive  constant  independent 
n 

of  n,  but  in  its  different  appearance,  it  may  take  different 
value,  e.g.  this  b  is  different  from  that  b  in  (3.13). 

From  (3.5),  (3.15)  and  (3.16),  it  follows  that 

P( |L  -T  l*2se*)  s  4s  exp {-bn}.  (3 

n  n 1 

Finally,  we  shall  establish  the  similar  bound  for 
P ( | Tn-R| 23se ' ) .  According  to  Lusin's  Theorem,  for  each  i, 
i  *  l,2,...,s,  there  exists  a  continuous  function  P^(x) 
satisfying  the  following  two  conditions 

a)  Os  P.(x)  s  sup  P. (x)  s  1,  (3 

x 

b)  F  (A)  <  5/2,  A  *  { X£  Rd ,  Pi(x)  ¥  P^X)},  (3 

where  5  ’is  the  constant  determined  by  inequality  (3.13). 
Define 

=  { jsn,  X.«A}. 
n  3 

Then  we  have 

P(1j!1Pi(X3)Qnj  “  J1Pi(Xj)Qnjl2£,) 


P(  l 
j  e 


Q  .  2E 
nj 


)  s  P{  Z  u  .  s  e') 


n 


j«*f 


nj 


s  P(#{?_}  a  5n)  +  P(E_)  . 


(3 


By  Hoeffding’s  inequality  (see  Hoeffdmg  (1963)),  we  have 

P(#{?n>  Hn)  s  2  exp{-n (6  -  F(A) )  2/(2F(A)  +  (6  -  F (A) ) )  } 


s  2  exp{-(|-)n}  *  2  exp{-bn}. 


(3.21 


(3.21),  together  with  (3.14),  yields 


P(|  E  P<(X.)Q  .  -  Z  P .  (X.)Qn  .  |2£’)  s  3  exp  {-bn} 
j»l  1  J  nJ  1  j  “J 


Define 


H.  -  [  P. (x)P.(x)Fi 
1  J  h  1  1 


(3.22 


(3.23 


[i  ”  |  P  : 
J  d*i 


(x)F(dx)  . 


(3.24 


It  is  obvious  that 


|H.  -  H. |  s  F(A)  <  5/2  <  e' . 

J*  1 


(3.25 


On  the  other  hand,  using  the  same  approach  as  in  Fritz  (1975) , 


we  can  prove  that 

P  ( |  Z  P .  (Xi )  -  H .  |  *  e' )  s  e“bn 
j-1  1  J  1  £ 

From  (3.22),  (3.25)  and  (3.26),  it  follows  that 


(3.26 


P(|^iPi(Xj)Qnj  -  Hi|  a  3e ' )  *  (3  +  p-)e~bn. 


(3.27 


Recalling  that  R  *  1  -  Z  H .  we  obtain  that 
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(3.28) 


p(|Tn-R|  a  3se*)  s  s(3  +  ^-)e"bn 

(3.17)  and  (3.28)  yield 

P(|Ln-R|a5Se')  s  S(7  +  ^r)e”bn,  (3.29) 

which,  together  with  taking  e'  =  e/5S,  completes  the  proof 
of  Theorem  3.1. 
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